ControlCom: Controllable Image Composition using Diffusion Model

Image composition targets at synthesizing a realistic composite image from a pair of foreground and background images. Recently, generative composition methods are built on large pretrained diffusion models to generate composite images, considering their great potential in image generation. However, they suffer from lack of controllability on foreground attributes and poor preservation of foreground identity. To address these challenges, we propose a controllable image composition method that unifies four tasks in one diffusion model: image blending, image harmonization, view synthesis, and generative composition. Meanwhile, we design a self-supervised training framework coupled with a tailored pipeline of training data preparation. Moreover, we propose a local enhancement module to enhance the foreground details in the diffusion model, improving the foreground fidelity of composite images. The proposed method is evaluated on both public benchmark and real-world data, which demonstrates that our method can generate more faithful and controllable composite images than existing approaches. The code and model will be available at https://github.com/bcmi/ControlCom-Image-Composition

Diff-ID: An Explainable Identity Difference Quantification Framework for DeepFake Detection

Despite the fact that DeepFake forgery detection algorithms have achieved impressive performance on known manipulations, they often face disastrous performance degradation when generalized to an unseen manipulation. Some recent works show improvement in generalization but rely on features fragile to image distortions such as compression. To this end, we propose Diff-ID, a concise and effective approach that explains and measures the identity loss induced by facial manipulations. When testing on an image of a specific person, Diff-ID utilizes an authentic image of that person as a reference and aligns them to the same identity-insensitive attribute feature space by applying a face-swapping generator. We then visualize the identity loss between the test and the reference image from the image differences of the aligned pairs, and design a custom metric to quantify the identity loss. The metric is then proved to be effective in distinguishing the forgery images from the real ones. Extensive experiments show that our approach achieves high detection performance on DeepFake images and state-of-the-art generalization ability to unknown forgery methods, while also being robust to image distortions

How Fragile is Relation Extraction under Entity Replacements?

Relation extraction (RE) aims to extract the relations between entity names from the textual context. In principle, textual context determines the ground-truth relation and the RE models should be able to correctly identify the relations reflected by the textual context. However, existing work has found that the RE models memorize the entity name patterns to make RE predictions while ignoring the textual context. This motivates us to raise the question: ``are RE models robust to the entity replacements?'' In this work, we operate the random and type-constrained entity replacements over the RE instances in TACRED and evaluate the state-of-the-art RE models under the entity replacements. We observe the 30\% - 50\% F1 score drops on the state-of-the-art RE models under entity replacements. These results suggest that we need more efforts to develop effective RE models robust to entity replacements. We release the source code at https://github.com/wangywUST/RobustRE

O2ATH: An OpenMP Offloading Toolkit for the Sunway Heterogeneous Manycore Platform

The next generation Sunway supercomputer employs the SW26010pro processor, which features a specialized on-chip heterogeneous architecture. Applications with significant hotspots can benefit from the great computation capacity improvement of Sunway many-core architectures by carefully making intensive manual many-core parallelization efforts. However, some legacy projects with large codebases, such as CESM, ROMS and WRF, contain numerous lines of code and do not have significant hotspots. The cost of manually porting such applications to the Sunway architecture is almost unaffordable. To overcome such a challenge, we have developed a toolkit named O2ATH. O2ATH forwards GNU OpenMP runtime library calls to Sunway's Athread library, which greatly simplifies the parallelization work on the Sunway architecture.O2ATH enables users to write both MPE and CPE code in a single file, and parallelization can be achieved by utilizing OpenMP directives and attributes. In practice, O2ATH has helped us to port two large projects, CESM and ROMS, to the CPEs of the next generation Sunway supercomputers via the OpenMP offload method. In the experiments, kernel speedups range from 3 to 15 times, resulting in 3 to 6 times whole application speedups.Furthermore, O2ATH requires significantly fewer code modifications compared to manually crafting CPE functions.This indicates that O2ATH can greatly enhance development efficiency when porting or optimizing large software projects on Sunway supercomputers.Comment: 15 pages, 6 figures, 5 tables

Pattern and Predictive Factors of Metastasis in Lymph Nodes Posterior to the Right Recurrent Laryngeal Nerve in Papillary Thyroid Carcinoma

ObjectiveThe right cervical central lymph nodes include lymph nodes anterior to the right recurrent laryngeal nerve (LN-arRLN) and lymph nodes posterior to the right recurrent laryngeal nerve (LN-prRLN), and are separated by the right recurrent laryngeal nerve (RLN). LN-prRLN is a common site of nodal recurrence after the resection of papillary thyroid carcinoma (PTC). However, the complexity in anatomical structure brings difficulties in determining the surgical scope, so it is necessary to assess the pattern and predictive factors of right cervical central lymph nodes, especially LN-prRLN metastasis in papillary thyroid carcinoma.MethodsA total of 562 diagnosed PTC patients who underwent right or total thyroidectomy were enrolled in this retrospective study. The clinicopathological features were collected, univariate and multivariate analyses were performed to determine predictive factors of the right central lymph node metastasis.ResultsIn this study, the metastatic rates of the right CLN, the LN-arRLN and the LN-prRLN were 59.6% (335/562), 51.8% (291/562) and 30.4% (171/562), respectively. And 22.6% (127/562) of patients had both LN-arRLN and LN-prRLN metastasis. Among patients without LN-arRLN metastasis, the rate of LN-prRLN metastasis was 16.2% (44/271), accounting for 25.7% of the LN-prRLN metastasis group. Factors associated with an increased risk of LN-arRLN metastasis include male, age below 55 years, tumor size > 1cm, extrathyroidal extension (ETE), clinical lymph nodes metastasis(cN1), lateral lymph node metastasis, and left CLN metastasis. In addition, ETE, lateral lymph node metastasis, and LN-arRLN metastasis were independent factors of LN-prRLN metastasis. The predictive factors of LN-prRLN in cN0 PTC were further explored, revealing that tumor size ≥1.5cm, ETE, and LN-arRLN metastasis were independent predictors of LN-prRLN metastasis in cN0 PTC.ConclusionThe LN-prRLN should not be ignored in surgery because of its high rate of metastasis. Our findings indicate that thorough dissection of central lymph nodes, especially LN-prRLN is crucial in clinical work

Space-based formaldehyde measurements as constrains on volatile organic compound emissions in east and south Asia and implications for ozone

We use a continuous 6-year record (1996–2001) of GOME satellite measurements of formaldehyde (HCHO) columns over east and south Asia to improve regional emission estimates of reactive nonmethane volatile organic compounds (NMVOCs), including isoprene, alkenes, HCHO, and xylenes. Mean monthly HCHO observations are compared to simulated HCHO columns from the GEOS-Chem chemical transport model using state-of-science, “bottom-up” emission inventories from Streets et al. (2003a) for anthropogenic and biomass burning emissions and Guenther et al. (2006) for biogenic emissions (MEGAN). We find that wintertime GOME observations can diagnose anthropogenic reactive NMVOC emissions from China, leading to an estimate 25% higher than Streets et al. (2003a). We attribute the difference to vehicular emissions. The biomass burning source for east and south Asia is almost 5 times the estimate of Streets et al. (2003a). GOME reveals a large source from agricultural burning in the North China Plain in June missing from current inventories. This source may reflect a recent trend toward in-field burning of crop residues as the need for biofuels diminishes. Biogenic isoprene emission in east and south Asia derived from GOME is 56 ± 30 Tg yr−1, similar to 52 Tg yr−1 from MEGAN. We find, however, that MEGAN underestimates emissions in China and overestimates emissions in the tropics. The higher Chinese biogenic and biomass burning emissions revealed by GOME have important implications for ozone pollution. We find 5 to 20 ppb seasonal increases in surface ozone in GEOS-Chem for central and northern China when using GOME-derived versus bottom-up emissions. Our methodology can be adapted for other regions of the world to provide top-down constraints on NMVOC emissions where multiple emission source types overlap in space and time.Earth and Planetary SciencesEngineering and Applied Science